Temporal concatenation for Markov decision processes

We propose and analyze a temporal concatenation heuristic for solving large-scale finite-horizon Markov decision processes (MDP), which divides the MDP into smaller sub-problems along time horizon generates an overall solution by simply concatenating optimal solutions from these sub-problems. As “black box” architecture, works with wide range of existing algorithms. Our main results characterize regret compared to solution. provide upper bounds general instances, as well family instances in are shown be tight. Together, our demonstrate concatenation's potential substantial speed-up at expense some performance degradation.